System and methods for calibrating a digitizer system

ABSTRACT

Systems and methods for calibrating a photoluminescent indicia-based digitizer system with a display, the photoluminescent indicia uniquely identifying local areas of a substrate.

BACKGROUND

Users are increasingly demanding functionalities beyond merelyrecognizing a touch to the surface of the touch-sensitive device. Suchother functionalities include handwriting recognition and direct notetaking (using, for example, a stylus). Such functionalities aregenerally provided in so-called digitizing systems.

Digitizing systems having position-dependent indicia detected by animage sensor in a stylus are commercially available. Anoto Group ABsells a stylus that detects indicia printed on an opaque paper orcardboard substrate. Reference is made to US Patent Publication No.2010/0001962 (Doray), which describes a multi-touch display system thatincludes a touch panel having a location pattern included thereon.

SUMMARY

Methods and systems for calibrating a digitizer system that is basedupon photoluminescent indicia that define local areas of a substrate,with an associated display. A localized area of the display isenergized, e.g., a dot is shown on the display. A user then selects thedot with the stylus. In one embodiment, the radiation from the dotitself causes at least one photoluminescent indicium to luminesce. Inanother embodiment, the stylus includes a radiation source which causesat least one photoluminescent indicium to luminesce. Either way, suchluminesced (emitted) radiation is sensed by the stylus. A processor thenassociates the sensed indicium with the displayed dot. This process maybe repeated as needed to accurately calibrate the substrate to thedisplay.

In one embodiment, a method of calibrating a digitizer system having asubstrate with photoluminescent indicia that uniquely identify localareas of a substrate, with a display is described, the methodcomprising: receiving, into an electronic sensing device, radiation froma first localized area of the display and radiation fromphotoluminescent indicium associated with the first localized area.

In another embodiment, a digitizer system is described, the systemcomprising an electronically addressable display; a substrate havingphotoluminescent indicia that uniquely define local areas of thesubstrate, wherein the substrate is coupled to the display; a sensingunit that senses the photoluminescent indicia; a processorcommunicatively coupled to the display and the sensing unit; wherein theprocessor causes a first localized are of the display to radiate whilereceiving signals indicative of radiation input received from thesensing unit sensing at least one photoluminescent indicium associatedwith the first localized area; and, wherein the processor computescalibration data based on the received signals, the calibration dataassociating localized areas of the substrate to the display.

These and other embodiments are described further herein.

BRIEF SUMMARY OF DRAWINGS

Embodiments described herein may be more completely understood inconsideration of the following detailed description in connection withthe accompanying drawings, in which:

FIG. 1a shows a digitizer and display system.

FIG. 1b shows a further embodiment of a digitizer and display system.

FIG. 1c shows a further embodiment of a digitizer and display system.

FIG. 1d shows an embodiment of a digitizer absent a display system.

FIG. 2a shows a magnified drawing of a prior art position-unique indiciapattern.

FIG. 2b shows a magnified drawing of a 6×6 position-unique indiciapattern with a plurality of wavelength combinations.

FIG. 2c shows a magnified drawing of a 5×5 position-unique indiciapattern with a plurality of wavelength combinations.

FIG. 2d shows a filter that could be used conjunction with aposition-unique indicia pattern with a plurality of wavelengthcombinations.

FIG. 3a shows a cross section of the end portion of a stylus.

FIG. 3b shows a cross section of the end portion of dual-source stylus.

FIG. 3c shows a cross section of the end portion of dual-source styluswith a coaxial optical path and light guide.

FIG. 3d shows a cross section of the end portion of dual-source styluswith a coaxial optical path, dichroic mirror, and two image sensors.

FIG. 3e shows a cross section of the end portion of a stylus similar theone shown in FIG. 3D, with two image sensors having filters.

FIG. 4a shows a magnified cross section view of a portion of a digitizerand display system.

FIG. 4b shows a magnified cross section view of a portion of a digitizerand display system with a dichroic reflector.

FIG. 4c shows a magnified cross section view of a portion of a digitizerand display system.

FIG. 4d shows a magnified cross section view of a portion of a digitizerand display system.

FIG. 4e shows a magnified cross section view of a portion of a digitizerand display system with a minimal overlay.

FIG. 4f shows a magnified cross section view of a portion of a digitizerand display system with stylus operating in reflective mode.

FIG. 4g shows a magnified cross section view of a portion of a digitizerand display system with indicia printed on the color filter of adisplay.

FIG. 5a shows an exemplary product construction.

FIG. 5b shows an exemplary product construction.

FIG. 5c shows an exemplary product construction.

FIG. 5d shows an exemplary product construction.

FIG. 5e shows an exemplary product construction.

FIG. 5f shows an exemplary product construction.

FIG. 6 shows an indicia-printed overlay on a display with a calibrationindicium illuminated on the display.

In the figures, like reference numerals designate like elements, unlessotherwise described.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

A digitizer system is described herein that includes a substrate havingphotoluminescent indicia that uniquely define an X-Y area of thesubstrate, and a pen or stylus with optical sensor that sense theindicia and determine based on the sensed indicia the position of thestylus relative to the substrate.

One pattern of indicia suited to embodiments described herein has beendeveloped by, and is available from, Anoto Company AB, of Sweden. Anotoprovides companies the ability to print small, opaque dots on paper in apattern that uniquely defines each location on the paper. A stylus, thatthey also provide, is then used to sense portions of the pattern passingwithin the stylus field-of-view (FOV). The sensed pattern is thenanalyzed and the position of the stylus relative to the paper iscomputed.

Such analysis may comprise subjecting the sensed pattern to an imagerecognition algorithm whereby the sensed image is analyzed with amathematical function that defines shapes or patterns, and/or withcompared against a library of indicia. Within such a library aredefinitions of a large number of supported indicia and an indicator ofthe relative location of each indicium relative to other indicia.Identifying one indicium provides an indication of location on thesurface of a digitizer. If digitizer locations have been predeterminedrelative to display coordinates, then indicia can be used to indirectlyreference display coordinates. Identification of orientation (forexample the rotation) of indicia relative to a stylus may also provideinformation about the orientation of the stylus.

As described further herein, the use of photoluminescent indicia, insome embodiments, allows for the substrate and the indicia to be highlylight transmissive to human visible light, even to the point of beingnearly transparent, such that it is well suited for use as a transparentoverlay to be put on or incorporated into a display. Photoluminescentindicia could also be used on non-light-transmissive substrates,including opaque substrates (such as whiteboards or paper). Amulti-wavelength pattern is also described herein.

Photoluminescent indicia also, in some embodiments, may improve thesignal to noise ratio in a detection system. They may also provideimproved handling of specular reflection from the substrate. In someembodiments, radiation from the excitation source can be reduced oreliminated at the detector by the use of an optical filter, therebyimproving the detection of the photoluminescent qualities of theindicia. In particular, since photoluminescent indicia, in someembodiments, upon receiving excitation electromagnetic illumination (orradiation—the terms are used interchangeably herein) in a firstwavelength range (usually in the form of ultraviolet (UV), visible, orinfrared (IR) illumination), the indicia luminesce providing emittedelectromagnetic radiation in one or more wavelength ranges differentthan the excitation range.

In some embodiments, signals associated with the excitation radiationsource can be filtered out, thereby increasing the signal-to-noise ratioin the light received by the sensor from the luminescent indicia. Thematerial of the substrate itself may also be selected, in someembodiments, to improve signal-to-noise ratios. For example, in aphotoluminescent indicia detection system, the substrate with theindicia could be placed on an absorbing material, a diffusing material,or a transparent material in order to minimize excitation radiationreturning to the sensor.

The material comprising the photoluminescent indicia may be selectedbased on properties of the substrate, as will be described more fullylater. In some embodiments, the material may comprise photoluminescentinks available in the marketplace, or quantum dots (QDs). Thephotoluminescent material is configured to exhibit photoluminescence; inone embodiment the luminescent emissions provided by the indicia isprimarily in the infra-red (IR) light wavelengths, and the “stimulating”excitation illumination is primarily in wavelengths shorter than thoseassociated with IR. There may be some overlap in the light wavelengthranges used to excite the indicia as compared with that provided by theemission of the material, but in some embodiments less overlap isdesirable. The photoluminescent material may be transmissive ortransparent to visible light, and may be disposed on a substrate thatitself is transmissive or transparent to visible light.

In some embodiments, the digitizer system may be configured to operatein one of several modes. In one mode, the stylus illuminates a field ofview with light in a first illumination wavelength range that excitesthe photoluminescent indicia, and images are detected by an imagedetector, or sensor, that is responsive to emission wavelengths in asecond indicia wavelength range that is substantially different from thefirst illumination wavelength range. In a second mode, the stylusprovides illumination in a wavelength range, and the stylus detectsnon-photoluminescent images in the same wavelength range as the stylusillumination. In a third mode, the stylus may detect indicia radiatinglight in a first indicia wavelength range from a visible light emittingdisplay. In some embodiments, the stylus image sensor and stylusprocessor may sense two or more wavelength ranges and discriminatebetween images in each wavelength range.

FIG. 1a shows stylus digitizer system 100 comprising stylus 120, display105, digitizer panel 115 patterned with photoluminescent indicia, andelectronic controller 130 that controls displayed images via link 135.Controller 130, comprised of various microprocessors and circuitry,controls displayed images via link 135, or it may be communicativelycoupled to another display-specific controller that controls displayedimages. Display controller 130 receives signals from stylus 120associated with the position of stylus 120 relative to digitizer panel115. Controller 130 may also serve as the system processor for acomputer system, for example a portable computing system. Thoughcontroller 130 is shown as one box in FIG. 1, it may exist as discreteelectronic sub-systems, for example, a first sub-system to interact withstylus 120, and a second sub-system to interact with display 105. Stylus120 is communicatively coupled with controller 130 by link 124. Link 124may be a bundle of thin wires, but more preferably link 124 is awireless radiofrequency (RF) link, and in such a construction stylus 120and controller 130 include radios for back-and-forth communication, or,depending on implementation, one way communication. Such radios in oneembodiment implement the Bluetooth™ communications protocol, or thatdefined by IEEE 802.11.

Another electronic subsystem could be configured to time the excitationsource and the sensing unit (for example, stylus). For example, theexcitation source could be pulsed (on/off) and the sensor capture timingset to correspond to the off state of the source. This configurationcould be useful in some embodiments for indicia based on phosphorescentmaterial or other photoluminescent material with a suitably long decaytime.

In some embodiments, pulsed operation of the source/detection systemwould also mitigate motion induced artifacts in a moving stylus system.If the capture time were sufficiently short, the blurring of the indiciain the captured image would be minimal, possibly allowing for moreaccurate reading of the indicia. In addition, there may be a reductionin the rate of photoluminescent bleaching as compared to operation ofthe source in a continuous mode. Pulsed mode operation may also extendthe operating time for battery powered stylus devices.

Stylus 120 has an optical image sensor that can detect patterns of lightwithin its field of view (FOV). Stylus 120 detects light 5 emitted fromphotoluminescent indicia disposed on or within digitizer panel 115.Stylus 120 may provide stimulating, or excitation, illumination in theform of excitation light 3 to illuminate indicia on digitizer panel 115.In other embodiments, excitation illumination may come from sourcesother than those housed within the stylus (for example, LCD backlightsand ambient light). Excitation light 3 may have a first wavelengthrange; indicia emitted light 5 (luminescence) has a second wavelengthrange. The first and second wavelength ranges in one embodiment do notoverlap. In another embodiment the first and second wavelength rangesminimally overlap such that most of the excitation illumination energyis at different wavelengths than most of the indicia emitted energy. Inanother embodiment, the first and second wavelength ranges overlap. Inyet another embodiment, emitted light 5 may be emitted from a visibledisplay so no excitation light 3 may be required. In some embodiments,the second wavelength range of light 5 will comprise a plurality ofwavelength combinations that may be discriminated from one another bythe optical image sensor in stylus 120. In most cases where indicia arefluorescent or phosphorescent, the first illumination wavelength rangewill be at shorter wavelengths than the second indicia wavelength range.The breadth of the first and the second wavelength ranges will be basedon the nature of the photoluminescent indicia. The first and/or thesecond wavelength ranges may be beyond those associated with humanocular sensing.

Display 105 may be any type display including but not limited toelectronically addressable displays such as liquid crystal displays(LCD), active matrix LCD (AMLCD), organic light emitting diode displays(OLED), active matrix organic light emitting diode displays (AMOLED),electrophoretic display, a projection display, plasma displays, or aprinted static image. Display 105 is, in some embodiments, optional, asthe digitizer may be used in applications where digitizer panel 115 isopaque.

Digitizer panel 115 is in one embodiment a transparent substrate, suchas glass, polyethylene terephthalate (PET), polyethylene naphthalate(PEN) cellulose triacetate (TAC), or any suitable material. It may befully light transmissive, partially light transmissive, or opaque.Preferably, it is highly light transmissive so as to allow a person tosee the output of display 105. Digitizer panel 115 may be one layer ormay be comprised of multiple layers of various materials. Digitizerpanel 115 includes, disposed upon or within it, photoluminescent indiciathat uniquely define the substrate or some portion of the substrate, intwo dimensions. Digitizer panel 115 may comprise more than one layer.For example, a durable layer may be used on the top surface.Anti-reflective (AR), anti-glare (AG), polarizing, color filtering,light reflecting, or dichroic optical layers may be included. Touchscreen electrodes or resistive surfaces may be included, as well asadhesive layers used to laminate various layers of panel 115. Panel 115may be rigid or flexible.

Anoto Company AB, of Sweden, licenses software that allows companies toprint opaque indicia in the form of ink on, for example, paper. Anotoalso sells pens that recognize the indicia and thereby determine the X-Ycoordinates of the pen relative to the printed paper.

Indicia and sensing techniques based on various patterns are furtherdescribed in U.S. Pat. Nos. 5,051,763; 5,442,147; 5,477,012; 5,852,434;6,502,756; 6,548,768; 6,570,104; 6,586,688; 6,663,008; 6,666,376;6,667,695; 6,689,966; 6,722,574; and 7,622,182, each of which is herebyincorporated by reference in its entirety. Anoto is one company that hasdeveloped location unique indicia-based digitizer systems; other systemswill be known by the skilled artisan, and inventions described hereinmay be applicable to many of them.

For digitizers that operate overlaid upon a display, indicia thatproduce indicia emitted light outside the visible range may bepreferable. Indicia for such applications are preferably formed of anysuitable material that provides emitted photoluminescence at wavelengthsbetween 700 and 1000 nm. Such materials are readily available, as are IRfilters and optical sensors that operate in this range. For example,Hamamatsu Photonics of Hamamatsu City, Japan, sells severalcharge-coupled device (CCD) optical sensors sensitive to IR wavelengthranges. For some applications, other wavelength ranges, such as longerIR wavelength ranges, may be preferable.

Any suitable photoluminescent material may be used for indicia. In oneembodiment, a suitable indicia material comprises photoluminescent inks.Some example photoluminescent inks and dyes are available from QCRSolutions Corp, of Port St Lucie Fla. (see dyes including IRF820A andIRF940A).

In another embodiment, photoluminescent quantum dots may be embedded ina carrier material, such as a resin or liquid, to make a dye. Quantumdots, in some embodiments, luminesce when exposed to excitation lightprovided over a wider range of wavelengths from UV to IR, which is nottrue for many other luminescent materials. Thus, quantum dots may beparticularly suited for, for example, system 102 (FIG. 1c and FIG. 4d )where a quantum dot luminescent material can absorb energy from a whiteLCD backlight. Other luminescent materials may require a specialbacklight for system 102 that emits light in their specific absorptionrange.

There exists a wide variety of commercially available quantum dotoptions. Quantum dots can be selected as to provide indicia emittedlight with a variety of wavelengths from the ultraviolet (for example,quantum dots comprised of ZnSe) through the visible (for example,quantum dots comprised of CdSe), and into the mid IR (>2500 nm) (forexample, comprised of PbSe. Quantum dots that provide indicia emittedlight in the IR range may also be made of PbS, PbSe, or InAs. Quantumdots made of PbS having diameters from about 2.7 to 4 nm will provideindicia emitted light in the near IR range of wavelengths. Quantum dotswith a core of InAs and a shell of higher band gap material, for exampleZnSe or PbSe with an inorganic passivation shell of CdSe, may haveimproved photoluminescence quantum efficiencies. Quantum dots havingcore of CdTeSe with a ZnS shell are commercially available from NanoOptical Materials of Torrance, Calif. Quantum dots stabilized with acombination of thioglycerol and dithioglycerol have also been shown toimprove stability of luminescence wavelength over time. Quantum dots maybe purchased commercially from companies including Nano OpticalMaterials, NOM's parent company Intelligent Optical Systems (also ofTorrance, Calif.), and Evident Technologies of Troy, N.Y. EvidentTechnology markets a PbS based dye with the name “Snake Eye Red 900”that may be formulated into an ink for printing. Life Technologies, ofGrand Island, N.Y., and Nanosys, of Palo Alto, Calif. offer printablequantum dot solutions, for example anti-counterfeit inks containingquantum dots.

Indicia may be printed onto any suitable substrate, as mentioned above.Many known printing processes may be used, including flexographic,gravure, ink jet, or micro-contact printing. Photoluminescent materialmay be dispersed homogeneously into an optically clear resin anddeposited on a substrate using methods such as flexographic or gravureprinting. Gravure printing may be preferable for printing largerphotoluminescent particles such as IRF820A. Flexographic printing mayhave cost advantages in high volume applications. Inkjet printing may bepreferred where smaller volumes of customized patterns of indicia aredesired. Inkjet printing of quantum dots is described in the articles:Small, A. C., Johnston, J. H. and Clark, N., Inkjet Printing of Water“Soluble” Doped ZnS Quantum Dots, European Journal of InorganicChemistry, 2010: pp. 242-247.

Printing of indicia in a thin layer is generally preferable inembodiments where indicia are printed on a substrate that overlays adisplay, because refraction of light through the indicia can beminimized. Transparency can also be maximized by thinner layers and insome cases luminescence quenching can be reduced. Photoluminescentquantum dots can be printed in a monolayer using micro-contact printing.A monolayer of quantum dots maximizes transparency to visible and IRwavelengths. In doing so, visible light transmission is maximized alongwith reflection of indicia-generated IR light (5 a, 6 a in FIGS. 4b, 4d).

Monolayers of quantum dots may be printed using micro-contact printingmethods described in the article Direct Patterning of CdSe Quantum Dotsinto Sub-100 nm Structures (Small, A. C., et. al., European Journal ofInorganic Chemistry (2010): pp. 242-247), and the article Fabricationand Luminescence of Designer Surface Patterns with β-CyclodextrinFunctionalized Quantum Dots via Multivalent Supramolecular Coupling(Dorokhin, et. al., Institute of Materials Research and Engineering,Agency for Science, Technology and Research, ACS Nano (2010), 4 (1), pp.137-142. The latter of these articles describes two methods ofmicrocontact printing luminescent CdSe/ZnS core-shell quantum dots. Inboth methods, quantum dots are functionalized by coating with surfaceligands of β-cyclodextrin that promote binding, and stabilize thequantum dots in a water-based colloidal suspensions. In both methods,polydimethylsiloxane (PDMS) stamps are used for micro-contact printingonto glass substrates. In one method, a substrate is firstmicrocontact-printed with a pattern of adamantyl terminated dendrimericmaterial, then a colloidal suspension of functionalized quantum dots areexposed to the printed substrate, and quantum dots bind to the materialprinted on the substrate. In a second method, functionalized quantumdots are directly microcontact printed onto a dendrimer layer on a glasssubstrate.

Depending on the photoluminescent pattern required, microreplication canbe used to make a tool with a negative of the desired pattern. The toolis then pressed and cured against a uniformly coated polymer layer toform consistent dents of the pattern in a polymer cured matrix. Thesedents can then be filled either directly with the a suitablephotoluminescent material blended with a carrier or formulated into anink using precision roll coating methods such as gravure printing, or,indirectly by using roll coating in combination with doctor-blading toremove excessive solution from areas other than within the dents.

Another way of patterning the substrate is to use a direct thermalprinting process with a photoluminescent ink so that the ink resideswithin the well formed in the (typically polymeric) substrate by thethermal printing mechanism. This may have the added advantage of rapidsingle step digital processing that can provide custom patterns without,in some embodiments, the need for costly tool development.

Photoluminescent indicia can be created using either organic orinorganic dyes or pigments depending on the nature of the application.Organic dyes, such as CY7 which is available from Lumiprobe ofHallandale, Fla., provide several benefits such as high luminescence.However, the Stokes shift for these types of dyes is typically <50 nmand the durability and light-fastness are often low, making themunsuitable for some applications. Carbon chains with conjugated bonds oraromatic rings are typical present in organic dyes, and are sometimesassociated with nitrogen or sulfur atoms. For example, CY7 consists of acyclohexane-bridged polymethyne chain.

Inorganic dyes, pigments, phosphors or other luminescent materials, suchas the earlier mentioned Snake Eye Red provide another solution. TheStokes shifts for these materials can be relatively high due to thelarge bandwidth of the absorption curves compared to organic dyes. Thesematerials consist of the cations of metal in an array with thenon-metallic ions, such as lead-sulfide (PbS) in Snake Eye Red.

Turning now to FIG. 1b , a stylus digitizer system 101 is shown,comprising stylus 120, digitizer panel 115 patterned withphotoluminescent indicia, display 105 a, and electronic controller 130that receives position-related information from stylus 120 and controlsdisplayed images via link 135. Display 105 a emits or reflects visiblelight, and display 105 a is at least partially transmissive to stylusemitted excitation light 3 and to indicia emitted light 5. Display 105Amay be a transparent OLED display, or static printed image, or otherdisplay type.

Turning now to FIG. 1c , a stylus digitizer system 102 is shown,comprising stylus 120, digitizer panel 115 with photoluminescentindicia, LCD 107, backlight 108, and electronic controller 130 thatreceives position-related information from stylus 120 and controlsdisplayed images via link 145. Stylus 120 detects patterns of light 5emitted from indicia on digitizer panel 115. Indicia emitted light 5preferably has a different wavelength range than visible light emittedfrom backlight 108. Indicia emitted light 5 preferably has a wavelengthrange including wavelengths sufficiently long such that they penetrateLCD 107 regardless of the On/Off state of the pixels in LCD 107. Forexample, 950 nm light penetrates most LCDs regardless of pixel state.Indicia on digitizer panel 115 may be energized by excitation light frombacklight 108, so an indicia exciting light source on stylus 120 may beoptional and not required.

FIG. 1d shows stylus digitizer system 103 comprising stylus 120,digitizer panel 115 with photoluminescent indicia, and electroniccontroller 130. Stylus 120 illuminates indicia on digitizer panel 115with excitation light 3 and detects indicia emitted light 5. Digitizerpanel 115 may be printed with various graphics, or it may appear blank(as for e.g. a blank sheet of paper), and may be opaque. Stylus 120 maybe combined with pen functionality (not shown in FIG. 1d , describedwith respect to inking tip 52, FIG. 4). Digitizer pane 115 may be awhiteboard, for example as used in a classroom, and the marker may beincorporated into the stylus, or the digitizer pane may comprise ascreen onto which a display is projected.

FIG. 3a shows a cross sectional view of a portion of stylus 120A. Stylusbody 41 contains optional light source 34 that may emit excitation light3 having a first illumination wavelength range. Indicia emitted light 5having a second indicia wavelength range enters the tip of stylus 120Aand is passed through filter 43. Filter 43 selectively passes at leastsome portion of the second wavelength range of indicia emitted light 5while blocking light of the first wavelength range (that was emitted bylight source 34). For example indicia may emit 800 nm to 1200 nm lightand filter 43 may pass light of indicia wavelengths between 750 nm and1200 nm while blocking light wavelengths below 750 nm. Lens 48 focusesindicia emitted light 5 to pass through aperture 33 then reflect frommirror 32 onto image sensor 45. Lens 48 may be made of IR transparent,visible light blocking material so lens 48 can also perform thefiltering function of filter 43. Exemplary lens 48 is shown as a simpleconvex lens, but other lens configurations may be preferable. In somestylus configurations, lens 48 may be required to focus a wide range ofwavelengths onto image sensor 53. Where this is the case, lens 48 may bean achromatic lens.

Image sensor 45 may be any suitable sensor. Sensors based oncharge-coupled devices (CCDs) and complementarymetal-oxide-semiconductor (CMOS) technologies are used interchangeablyin multiple areas of imaging, and may be suitable in this application.In some embodiments, light 5 may have a plurality of indicia wavelengthand pattern combinations that uniquely define local regions of asubstrate. In these embodiments, image sensor 45 may further include acolor filter that passes selected indicia wavelengths of light 5 to somepixels and different wavelengths to other pixels of sensor 45.

Image sensor 45 is connected to stylus processor 44 by conductors onprinted circuit board (PCB) 46. Light source 34 is connected to PCB 46 aby link 36. Stylus processor 44 controls image sensor 45 and lightsource 34 via conductors on PCB 46 a and by link 36 respectively. Inaddition, the stylus processor controls collection of image informationfrom image sensor 45 and communicating this to controller 130 via link124, (shown in FIG. 1a ). Stylus 120 may also comprise additionalcomponents such as switches and a battery, (not shown). Stylus 120 mayalso comprise a probe 51 extending from the stylus that provides spacingfor the optical components, and may activate a switch upon contact withthe surface of the digitizer, causing certain stylus electronics toactivate. Probe 51 may be made of solid plastic, or metal, and it maycontain ink for writing on a surface with stylus 120. Probe 51 may beoptionally retractable.

The illumination wavelength range of light source 34 must provideradiation in a range that excites indicia to produce emission(luminescence) at a desired indicia wavelength and brightness. In oneexample, indicia emitted light luminesces due to stimulation by light inanother, shorter wavelength range. In some embodiments it may bepreferable that the excitation light and the indicia emitting light beminimally visible to the user. These criteria are met if light source 34emits UV-A light, for example between 350 nm and 420 nm or if lightsource 34 emits near IR light, for example between 700 nm and 850 nm.Excitation light may instead, or in addition, be generated by othercomponents, such as backlights in an LCD panel, or other illuminationsources.

Stylus 120B (FIG. 3b ) is similar to stylus 120A, except it has anadditional light source 35. This light source may be beneficially usedin stylus embodiments that need to sense both photoluminescent indiciaand traditional reflective indicia (the latter commercially availablefrom Anoto). For example, in luminescent mode, processor 44 may selectlight source 34 to excite indicia with light in the 380 nM wavelengthrange that then luminesce and emit radiation in an indicia wavelengthrange centered on 850 nM. Image sensor 45 may be configured to detectimages in the indicia wavelength range of 850 nM. In reflecting mode,processor 44 may select source 35 to illuminate indicia that do notluminesce. Source 35 would preferably emit illumination in the indiciawavelength range of 850 nM, so light of this wavelength range will bereflected differently from the indicia relative to their surroundingsubstrate, and the resulting contrast between indicia-reflected lightand substrate-reflected light is detected by image sensor 45.

FIG. 3c shows a simplified cross sectional view of a portion of stylus120C that has two light sources like stylus 120A, but with analternative construction including a common optical path forillumination light 3 and 6, and indicia-emitted light 5. Stylus body 41c contains light sources 34 c and 35 c that emit illumination light 3light 6 respectively. Light sources 34 c and 35 c may have multipleemitters. In the example shown in FIG. 3c axial view, each sourcecomprises two LEDs. Also in the example, light from sources 34 c and 35c is focused by a Fresnel lens 63 in front of each of the four LEDs.Light from sources 34 c or 35 c is focused through objective lens 48 cand emitted from stylus 120C, where it may illuminate field of view 62.Light 5 within field of view 62 enters stylus 120C through lens 48 c andis focused through an aperture 33 in plate 56 c, then reflected onmirror 32. From mirror 32, light 5 passes to color filter 53 c wherelight 5 within a selected wavelength range is passed through filter 53 cto image sensor 45 c. The common optical path through lens 48 csimplifies alignment of outgoing light 3 or 6 with incoming light 5, andminimizes the required tip diameter of stylus 120.

Various operating modes of stylus 120 are described elsewhere herein forilluminating indicia with various wavelengths of light and sensingimages of photoluminescent or light-reflecting indicia. In a furthermode, stylus 120 may detect images of indicia that radiate light in thevisible wavelength range. For example, stylus 120 may sense imagesformed by pixels of a visible light emitting display, for example, asdescribed in U.S. Pat. No. 7,646,377, which is hereby incorporated byreference in its entirety. Sensing visible images from a light emittingdisplay such as an LCD, OLED, or projection display does not generallyrequire illumination by a stylus light source, so sources 34 and 35 maybe turned off by processor 44 when the stylus is in visible lightsensing mode. Sensing light in the visible wavelength range requiresthat image sensor 45 be sensitive to visible wavelengths, and thatincoming light 5 passes through filters so it reaches image sensor 45.Any of sensors 45 may be sensitive to visible light. Any of filters 43or 53, or similar, may pass visible wavelengths.

FIG. 3d shows a simplified cross sectional view of a portion of stylus120D that is similar to stylus 120C, but forward-facing light sources 34c and 35 c are replaced by lateral-facing light sources with a lightguide 61 that turns light 3 or light 6 by 90 degrees and focuses ittoward objective lens 48 d. Light guide 61 may be coated with reflectivematerial on its rear surface and on edges, with the exception of theedge where light from sources 34 d and 35 d enter light guide 61.Fresnel lens facets 62 on light guide 61 may be used to focus light 3and light 6 to uniformly illuminate the field of view 65. Light 5 entersthe end of stylus 120D through lens 48 d and is focused through aperture33, then reflected on mirror 32 d. From mirror 32 d, light 5 passes tocolor filter 53 d and to image sensor 45 d.

FIG. 3e shows a simplified cross sectional view of a portion of stylus120E that is similar to stylus 120D, but the single image sensor 45 d isreplaced by two image sensors 45 e and 75, having filters 53 e and 73,respectively. Light 5 enters the end of stylus 120E through lens 48 eand is focused through aperture 33 in aperture plate 56 d. Light in theIR wavelength range is then reflected on dichroic mirror 69 to filter 53e and image sensor 45 e. Filter 53 e may separate IR light intodifferent wavelength ranges so image sensor 45 e can discriminate onewavelength range from another, or image sensor 45 e may detect amonochrome image including a single range of IR wavelengths. Light 5 ein the visible wavelength range passes through dichroic mirror 69 andvisible light filter 73 to image sensor 75. Filter 73 may separatevisible light into different wavelength ranges so image sensor 75 candiscriminate one wavelength range from another, or image sensor 75 maydetect a monochrome image including the full range of visiblewavelengths. Flexprint 77 connects image sensor 75 to PCB 46 c.Processor 44 e uses image information from image detector 45 e and 75 toresolve images that may include images in an IR wavelength range and/orimages in a visible wavelength range. Dichroic mirror 69 may comprisethe same material used for dichroic substrate 148, described withrespect to FIG. 4b , separate image sensors may have advantagesincluding higher potential special resolution, and availability ofspecific ICs that lack a color filter, but have advantages such as lowcost and high imaging frame rate.

FIG. 2a shows an Anoto type indicium within the solid rectangle,comprising a pattern of opaque dots on a substrate, arranged on avirtual 6×6 matrix indicated by dotted lines. Each intersection of thematrix has one dot, and each dot is position-encoded into one of fourpositions above, below, left, or right of the intersection. Thisprovides a coding system based on thirty-six digits of base four, soeach 4-some combination of the indicia can represent as many as 4³⁶unique codes. Permutations of codes are reduced to allow independentencoding of X and Y coordinates in each indicium. Permutations are alsoreduced by the requirement for redundancy that allows determination ofthe position of a partial surface, and to detect code sequences from anyorientation. Even with such reductions, Anoto indicia can encodeextremely large areas with position-unique indicia less than 2 mm insize, where each of the indicia uniquely defines a local region of asubstrate.

Photoluminescent indicia described herein may use the same pattern ofmonochrome dots as used by Anoto, or other options may be preferable.For example, visible light transmissive photoluminescent materials maybe used to make features of larger size that still have minimal opticalvisibility. Larger dots or other feature shapes may be easier to print,and material with larger photoluminescent particles may be used, whileindicia size is maintained at less than 2 mm square.

Indicia features emitting distinguishable wavelengths of light provideadditional coding alternatives. The base four coding system describedabove can be achieved with four wavelength combinations instead offour-quadrant dot placements or four symbol shapes. An example is shownin FIG. 2b , where dots comprising photoluminescent nanoparticles withdifferent emission wavelength ranges are used to uniquely define anindicium 190, which together with other indicium may be used to uniquelydefine local areas of a substrate. A first indicia wavelength range maybe centered on 850 nM and a second indicia wavelength range may becentered on 950 nM. Indicia may comprise dots, or other shapes,positioned in any of thirty-six positions. Each possible dot positionhas one of four features: a dot of the first indicia wavelength range191, a dot of the second indicia wavelength range 192, a dot withindicia wavelengths from the first and second ranges 193, or no dot.Other wavelength combinations, and other feature shapes or combinationsof feature shapes may be used. For example, a four-position encodedpattern of dots may also have dots of two wavelength ranges, so eachposition in a 5×5 indicium has eight possible codes, resulting in morethan 4³⁶ unique codes in a 5×5 matrix.

FIG. 2c shows an indicium 195 comprising a 5×5 array that comprise dotsthat luminesce in a plurality of wavelength combinations, the wavelengthcombinations uniquely defining local regions of the substrate. The 25possible positions in the array have dots having one of four features: adot of the first indicia wavelength range 191, a dot of the secondindicia wavelength range 192, a dot with indicia wavelengths from thefirst and second ranges 193, or no dot. The pattern is restricted to nodots at three corners, as shown; all other positions of the top, right,and bottom edges have dots that emit in at least one wavelength. Thispattern provides easy recognition of indicia borders and angularorientation. Given feature-to-feature spacing of 0.3 mm, a repeatingpattern of unique indicia can provide pattern and wavelengthcombinations uniquely defining local regions of a substrate area nearly500,000 square meters.

FIG. 2d shows details of a portion of exemplary color filter 53 that isconfigured as an array of filter cells, using known color filtermethods. Most color filters have arrays of cells that pass red, green,or blue light to pixels an image sensor. For example many color filtersuse a Bayer filter (U.S. Pat. No. 3,971,065) that has twice as manygreen filter cells as red or blue, often in a pattern of R,G,G,B orR,G,B,G. Filter 53 replaces one or more of the R,G,B filter cells withcells that pass light in the IR wavelength range. Filter 53 can pass upto four wavelength ranges to selected pixels of an image sensor, so theimage sensor can discriminate among image features having differentwavelength ranges, or colors. In one embodiment, filter 53 passes twowavelength ranges of visible light and two wavelength ranges of IR lightto image sensor 45, with coupled electronics that resolves a locationbased on signals from the image sensor. Such a filter, or one like it,may be advantageously used in conjunction with resolving indicia such asindicium 190 and 195.

In one embodiment described with respect to FIG. 2b and FIG. 2c , fourcolor filter cells in filter 53 comprise cells 112 and 113 that pass IRlight centered on 850 nM and 950 nM respectively. And visible lightcells 114 and 115 that pass light centered on wavelengths 500 nM and 600nM respectively. IR cells 112 and 113 preferably pass light with abandwidth of +/−50 nM. Visible cells 114 and 115 preferably pass lightwith a bandwidth of +/−100 nM. Other color filter layouts may be used,and other filter wavelength combinations may be used. In someembodiments, it may be sufficient to detect monochrome visible images,so filters 114 and 115 may both detect visible light from 450 nM to 700nM, for example. In some embodiments, visible image detection may not berequired, so all cells of filter 53 may pass one or more IR wavelengthranges.

Table 1 summarizes the operating modes of stylus 120. Modes will bedescribed with respect to components of stylus 120C, although the modesapply to the other exemplary stylus configurations herein. Table 1 showsexamples of components used in various combinations to read images fromvarious media.

Stylus Mode 1—Photoluminescent Media

To read photoluminescent indicia, light source 35 is turned on andsource 34 is off. The indicia image may be filtered through anIR-transparent color filter. For example, filter 43 or cells of a colorseparating filter such as cells 112 and/or 113 in filter 53. The imageis read from image sensor 45 by processor 44.

Stylus Mode 2—Passive Media

To read light absorbing passive indicia on a light diffusing orreflective substrate such as paper, source 34 is turned on and source 35is off, so light 3 emitted from stylus 120 is at a wavelength that willpass through stylus filters 43 or 53 to stylus image detector 45. Light5 reflected from indicia having an image formed by contrast betweenindicia features and the substrate, is received and filtered throughIR-transparent color filter 43 or IR-transparent cells in filter 53, andthe IR image is read from image sensor 45 by processor 44.

Passive indicia on a light transparent substrate may absorb light, orthey may diffuse or reflect it. To read light from such media, source 34is turned on and source 35 off, so light 3 emitted from stylus 120 is ata wavelength that will pass through stylus filters 43 or 53 to stylusimage detector 45. The illuminated indicia image light 5 is received andfiltered, and the IR image is read from image sensor 45 by processor 44.The image received from passive indicia on a transparent substrate maybe a reverse image relative to passive media on an opaque substrate,depending on whether the indicia or the background reflect light 5toward stylus 120.

Stylus Mode 3—Visible Emitting Display

To read a light emitting visible image, (for example, indicia displayedon an LCD or OLED), source 35 and source 34 may be turned off. Adisplayed cursor may be used as a location-specific indicium, or pixelson a display may be tracked to detect movement as described in U.S. Pat.No. 7,646,377. In addition, the IR-measuring pixels of image sensor 45may be read, to detect any time-varying IR signals that may be emittedfrom the display. Time-varying IR signals may be encoded to indicatewhich of several displays are in the field of view of stylus 120, asdescribed in U.S. patent application Ser. No. 13/454,066, which ishereby incorporated by reference in its entirety.

TABLE 1 Stylus Component Image Image source 35 sensor sensor Image(excitation source 34 IR visible Mode being read light) (IR light)pixels pixels 1 Photoluminescent On Off Read Indicia image 2 PassiveIndicia Off On Read (reflective) image 3 Visible emitting Off Off Readdisplay image

In one embodiment, multi-mode stylus 120 can operate in various modes aslisted in Table 1. Manual switching of modes may be done with switcheson the stylus or by interaction with an application and GUI on adisplay. Automatic switching among modes may also be supported, based oninput received by stylus 120. For example, given stylus 120 with filter53 and image sensor 45 that can sense images in multiple wavelengthranges, and two illumination sources 34 and 35 of different wavelengths,images under different combinations of source illumination may be testedsequentially until a valid image is recognized. The following exemplaryalgorithm may be used to automatically switch stylus sensing modes:

-   1. Stylus Mode 3 is activated, whereby sources 35 and 34 are turned    off, then visible images are detected.    -   1.1. Detected images are processed to recognize any supported        indicia in the stylus FOV. If no supported indicium is detected,        go to step 2, otherwise,    -   1.2. Any detected visible image is correlated with supported        indicia or stylus-locator cursors, for example as, for example,        described in U.S. Pat. No. 7,646,377.        -   1.2.1. If matched, a display-referenced location is            calculated and reported.            -   1.2.1.1. Repeat from Step 1.1        -   1.2.2. If unmatched, report no stylus-related visible image            is in the FOV.-   2. Stylus Mode 1 is activated, whereby source 35 is turned on and    source 34 is turned off, then IR images are detected.    -   2.1. Any detected IR image is correlated with supported indicia        patterns. If a positive correlation is determined, a luminescent        digitizer location is calculated and reported.        -   2.1.1. Repeat Step 2 until no IR image is detected, then go            to Step 1-   3. Stylus Mode 2 is activated, whereby source 35 is turned off and    source 34 is turned on, then IR images are detected.    -   3.1. Any detected IR image is correlated with supported indicia        patterns. If a positive correlation is determined, a passive        digitizer location is calculated and reported.        -   3.1.1. Repeat Step 3 until no IR image is detected, then Go            to Step 1.

Referring now to FIG. 4A, photoluminescent indicia 119 are deposited onthe surface of visible light transmissive substrate 118. Indicia may bedeposited by printing a thin layer of photoluminescent material. Forexample photoluminescent quantum dots may be mixed with a printablemedium, and printed using known methods. Indicia-printed substrate 118is then laminated to durable overlay 116, which may be glass, acrylic,or any suitable transparent working surface that will protect theindicia. Alternatively, indicia may be printed on the bottom surface ofoverlay 116.

Adhesive 117 that binds substrate 118 to overlay 116 may be an opticallyclear adhesive (OCA) such as those sold be 3M Co. of St Paul, Minn.,USA. Adhesive 117 may be replaced by an air gap, (providing indicia 119have an environmentally protective coating) and air gap 109 may also bereplaced with an OCA.

Illumination from stylus 120 comprises excitation light 3 and 3 a whichhave a first wavelength range (for example, near UV-A or near IR). Light3 a that does not get absorbed by indicia 119 passes through substrate118, and is absorbed or reflected by layers below. Some of light 3 isabsorbed by indicia 119 which excites indicia 119 such to radiateindicia emitted light 5 in various directions, including into the stylushousing which includes optical sensing electronics, as describedearlier. Indicia emitted light 5, in this embodiment, is primarily in asecond wavelength range, in this case it is within the IR range. Visibledisplay light 7 and 8 is emitted from display 105. Some display light 8that hits indicia 119 will pass through the indicia. Some of the displaylight 2 may also be absorbed by the indicia 119 and will excite theindicia causing it to luminesce, resulting in light 6. Ambient light 9may also cause photoluminescence in indicia 119, resulting in indiciaemitted light 6.

Display 105 is shown as a planar cross section, but display 5 mayinclude other shapes. For example, the display may comprise a rearprojector and a rear projection panel.

FIG. 4B shows a digitizer system identical to that in FIG. 4a , exceptsubstrate 118 is replaced by dichroic substrate 148 that passes light ofsome the first wavelength range while reflecting light of the secondwavelength range, where wavelengths of the first range are typicallyshorter than wavelengths of the second. For example, substrate 148 maybe visibly transparent IR reflective material such as 3M Company'smultilayer optical films (MOF) which are available commercially underthe names Crystalline automotive films and Prestige Series residentialwindow films. Dichroic reflective substrate 148 increases the efficiencyof the system by re-directing indicia-generated light of the secondwavelength range that would not otherwise have reached stylus 120. 3MCompany's PR90EX film may be used for the reflective substrate 148.

FIG. 4C shows a magnified cross section view of a portion of digitizerand display system 102 (see FIG. 1c ). In this embodiment, indicia areexcited by light radiation provided by backlight 108. In otherembodiments, however, the stylus could additionally include such afurther illumination source.

If display 107 is a light emitting display such as a (transparent) OLED,backlight 108 may not be required. Where optional backlight 108 is used(for example, in combination with an LCD display), visible (white)display light 7 and 8 may be emitted from optional display backlight108. A portion of light 7 that is not absorbed by Indicia 119A passesthrough substrate 118, optically clear adhesive layer 117, and throughpixels of display 107, to form a displayed image. Light from backlight108, may comprise wavelengths outside the visible spectrum. For example,near infrared (IR) light (700 nm-1000 nm) or near UV light (e.g. 350nm-400 nm) may be emitted from the backlight to energize indicia onsubstrate 118. Some of these wavelengths may be filtered out by layersabove the indicia, (for example, the color filter of display 107, whichwould be typical if 107 were an LCD), so minimal amounts of thesewavelengths of light may reach the user. An LCD display comprisingquantum dots energized by a backlight is described in US PatentApplication No. 2008/0246388, “Infrared Display with Luminescent QuantumDots.”

FIG. 4D shows a magnified cross section view of a portion of digitizerand display system 103 (see FIG. 1d ). The stylus, in this embodiment,is similar to that described with respect to FIG. 4b , in that itincludes an illumination source that excites the indicia. Thisembodiment shows the use of photoluminescent position-unique indicia ina whiteboard-type environment. A transparent overlay includingphotoluminescent indicia 119 and optically clear adhesive layer 117, aswell as substrate 156 (here, substantially light transmissive such thatgraphics on the surface of substrate 158 are visible through the stackcomprising substrate 156 and OCA 117). Surface 159 of substrate 156 maybe layered with a material, such as Tedlar® (available from DuPont),polypropylene, or other surface compatible with white-boardapplications, or it could be an anti-scratch, anti-reflective,anti-glare, polarizing, or color filtering layer. Substrate 156 maycomprise a plurality of layers. In such embodiment, stylus 120 mayinclude inks compatible with whiteboard applications, such that a usermay write on the whiteboard while electronics in the stylus computescoordinates of the writing and provides these to a computer.

FIG. 4E shows the use of photoluminescent indicia on an opaque ortranslucent substrate. Photoluminescent indicia 119 are deposited on thetop surface of substrate 168, which may be paper, cardboard, PET, PEN,glass, acrylic, or any material that supports indicia 119. Indicia 119are deposited by printing a thin layer of photoluminescent material. Forexample photoluminescent quantum dots may be mixed with a printablemedium, and printed using known methods. Indicia-printed substrate 168may then be covered with optional layer 167 that is transparent tostylus excitation light 3 and indicia emitted light 5. Layer 167 mayoptionally be added to protect indicia. Layer 167 may be an anti-scratchlayer, a durable layer comprising a polymer hardcoat, or a polymerhardcoat filled with silica particles, or a sheet of material such asPET may be laminated to substrate 168. Surface 169 may haveanti-reflective (AR), and/or anti-glare (AG) properties. Additionally,substrate 168 may be printed with a visible static image. While the penshowed in FIG. 4e includes an illumination source to excite indiciawithin the stylus FOV, depending on the application, the stylus couldsense indicia excited by other means, such as ambient light 9. In suchcase, the stylus would not necessarily need an illumination source.

FIG. 4F shows a somewhat different embodiment, where stylus 120 sensespassive indicia that operate in a light absorbing or light reflectingmode, for example, as indicated in Table 1, Mode 2. In this embodiment,stylus 120 provides an illumination source (for example, light source 34in FIG. 3c ) that is reflected by substrate 178, but not reflected by,or minimally reflected by the features of indicia 179, for example thedots exemplified in FIG. 2a . Indicia 179 preferably comprise a thinlayer of IR-absorbing material; substrate 178 is preferably anIR-reflecting or IR transparent material; and the image sensor in stylus120 senses the reflected light that has the same wavelength as theillumination source. An image is formed by contrast between reflectedlight and indicia-absorbed light. Thus this embodiment provides a“negative” image of the indicia, which may be desirable in someembodiments, for example to detect black (IR absorbing) indicia featuresprinted on white paper. Alternatively, indicia features may reflectlight of the illumination source 34 and the substrate may absorb stylusillumination, resulting in a “positive” image of indicia-emitted lighton a dark background being detected by image sensor 45.

FIG. 4G shows a magnified cross section view of a portion of digitizerand display system similar to system 100 (see FIG. 1A), except thesubstrate portion of digitizer 115 is eliminated and indicia 119 areprinted onto color filter 152 of LCD 105 g. Indicia 119 are in oneembodiment optically transparent, and the features of indicia 119 (forexample the indicia dot patterns similar to 190 or 195) may be alignedwith the color cells of color filter 152. As an alternativeconstruction, indicia 119 may be printed onto the bottom surface of toppolarizer layer 153. Integration into an LCD offers the advantage ofthinner construction and potential for better visible light transmissionthrough LCD 105 g.

Electronically addressable display 105 g (e.g., an LED, plasma, etc.)has display coordinates that uniquely identify the location of eachpixel. Similarly, digitizer 115 (FIG. 4A-4D) includes a digitizercoordinate system wherein each location on the surface is identified bylocation-unique indicia 119. Given the display is co-planar with adigitizer, digitizer coordinates can be aligned with display coordinatesof such systems using a calibration process that associates each displaypixel at a particular localized area with one or more of digitizerindicia 119 associated with that same localized area. After calibration,indicia 119 can be used to indirectly reference display pixelcoordinates and vice versa.

Calibration may be performed at a manufacturing site, or by the end userof a device. In some “after market” embodiments, a user may assemble thedigitizer 115 onto a display 105. In such case, it will generally benecessary for the user to perform calibration. The calibration systemdescribed below allows a user of minimal skill to perform an accuratecalibration. Two alternative calibration methods are described below.

FIG. 6 shows a view of display 105 and surface 115 with indicia 119,which are in this drawing represented by position-unique number andletter symbols (used herein only for illustration purposes). Inpractical systems, indicia will typically comprise position-unique dotor line patterns, for example those described with respect to FIG. 2.Each of the indicia 119 are located at known digitizer coordinates onsurface 115. The X,Y coordinates of indicia 119 are designated asiX_(n),iY_(m). Coordinates of pixels on display 105 are designated asdX_(n), dY_(m).

A first calibration embodiment is described with respect to FIG. 6. In acalibration mode, display 105 radiates a calibration indicium 10 at afirst localized area with a first indicia wavelength range thatcomprises visible light. In the example described with respect to FIG.6, indicium 10 is a round spot that serves as an indicium for stylus 120and may also serve as a user-visible cursor. Stylus 120, or genericallya sensor device or a sensing unit, is placed so calibration indicium 10is in its field-of-view (FOV) 65. Placement of the stylus may be done bya user aiming stylus 120 at visible indicium 10. Stylus 120 sensesindicium 10, and stylus 120 also illuminates its FOV with light of afirst illumination wavelength range and senses whatever photoluminescentindicia are also present with its FOV, which radiate light within asecond indicium wavelength range. In other words, the photoluminescentindicia being sensed are also associated with a common first localizedarea. In some embodiments, the second wavelength range is not the sameas the first. One or more digitizer indicia within the FOV will happento be co-located with display indicium 10. The digitizer will determineand note the coordinates of such co-located indicia. The co-locatedindicia are then associated with the display coordinates of thecalibration indicium. These coordinates are stored and used to relatesubsequent digitizer measurements to display coordinates. For example,in the example shown in FIG. 6, indicia r and s are in the FOV of stylus120 but only s is co-located with indicium 10 so s is associated withthe address of pixels forming indicium 10. Calibration may continue withindicium 10 placed at a second predetermined location, and theabove-described calibration procedure repeated.

A second embodiment of the calibration process is also described withrespect to FIG. 6. In this embodiment stylus 120 does not senseradiation in the first indicium wavelength range emitted by indicium 10,but only senses indicia-radiated light in a second indicia wavelengthrange that is emitted from digitizer indicia 119. Indicia 119 a, uponexposure to excitation radiation, luminesce in a second indiciawavelength range in response to light energy from indicium 10.

Display 105 activates indicium 10 having light in a first indiciawavelength range that may comprise visible light. Stylus 120 is placedso calibration indicium 10 is in its FOV 65. Light from pattern 10illuminates indicia 119 a, which is co-located with (i.e., in this casedirectly above) indicium 10, and causes them to radiate indicia emittedlight (luminescence) in a second indicia wavelength range that isdifferent from the first wavelength range of indicium 10. Stylus 120senses indicia 119 a in its FOV. Ambient light 9 is preferably limitedto prevent ambient light 9 from being a source of excitation radiationto indicia 119 that are not illuminated by light from indicium 10.

Display 105 is, in this second calibration embodiment, emitting no lightexcept for the displayed indicium 10 that is centered at displaycoordinates dX₁₀, dY₁₀. (which correspond to the center of indicium 10.Most of photoluminescent indicia 119 are dark because they have noexcitation energy, with the exception that indicium 10 is illuminatingindicia 119 a, causing it to luminesce. Stylus 120 with indicium 10 inits FOV senses indicia 119 a at digitizer coordinates iX₁₀, iY₁₀. Thusthe display coordinates dX₁₀, dY₁₀. are determined to be co-located withindicia coordinates iX₁₀, iY₁₀. Light sources 34 and 35 in stylus 120are turned off during this procedure of embodiment 2. Display 105 mayemit light in a time-modulated sequence and the stylus may demodulatethe resulting time-modulated signal re-radiated from indicia 119 a.

If display 105 has higher resolution than indicia 119, the location ofindicium 10 may be adjusted in size until it minimally circumscribes asingle indicium. This may increase the accuracy of aligning indicium 10with digitizer indicium 119 a. Indicium 10 may also be incrementallyadjusted in the X and/or Y direction as the illumination level of theexcited indicium 119 a is measured, to determine the exact location ofthe indicia relative to display 105.

The procedures described in both the first and second calibrationembodiments have a benefit that the calibration pattern must be in thestylus FOV, but it need not be in the center of the stylus FOV, soinaccurate placement of the stylus has minimal effect on calibrationaccuracy. Identification of orientation (for example the rotation) ofindicia relative to a stylus may also provide information about theorientation of the stylus relative to a digitizer and a display.

Using either calibration method described above, calibration data iscomputed by way of a processor, and such calibration data may then beoutputted to another computing device which may store such calibrationdata for future reference. For example, a computer may store thecalibration data so the calibration routine need not be repeated eachtime the computer boots.

Stylus 120 may have an extendable tip 51 that makes contact with adigitizer surface. In Mode 2, the stylus is used with passive indicia,which may include indicia printed on paper. Where writing on paper maybe preferable, an extendable/retractable inking tip 51 may be extended.In other modes, inking may not be desirable and a different (e.g.non-scratching) tip material such as Delryn plastic may be preferred.Stylus may have a plastic tip 71 that extends beyond lens 48, and anink-dispensing tip 51 that can be adjusted to extend beyond plastic tip71, or to retract so plastic tip 71 is the outermost point beyond lens48.

The digitizer panel can be integrated into a display stack-up in avariety of ways using a variety of rigid or flexible materials. Oneembodiment shown in FIG. 5a comprises an overlay where the digitizerpanel is attached to a device 205, in an after-market application usingself-wetting adhesive 201. This panel is constructed by forming thephotoluminescent indicia 202, on one side of substrate 200 (in this casePET), and covering these indicia with said adhesive 201. To attach thepanel, the user may place the adhesive-side of the panel onto the device205. To improve the durability of the overlay, a hard-coat may be addedto the side of the panel that faces the user.

Another embodiment shown in FIG. 5b for an overlay in aftermarketapplications may be constructed by first printing the indicia 202 onmultilayer optical film (MOF) 203, which reflects IR. In thisconstruction, the device-facing adhesive 204 may be placed on theopposite side as the indicia since the emitted signals from thephotoluminescent dyes do not pass through MOF 203. To further protectthe indicia from the user's interaction, a protective layer 200, such asPET, may be adhered to the top of the MOF to cover the indicia, usingadhesive 201.

Another embodiment is an underlay where the digitizer panel is attachedto a touch-sensitive screen including but not limited to those productofferings from 3M Touch Systems, as a surface-capacitive technology(SCT) screen or a projected-capacitive technology (PCT) screen. FIG. 5cshows such an embodiment, where the photoluminescent indicia 202, formedon a transparent substrate 200, such as PET or MOF, covered with anoptically-clear adhesive, 201. This adhesive may then be attached to theback of the touch sensitive screen 205, using a variety of methods knownin the art. Another embodiment as shown in FIG. 5d is to print thephotoluminescent indicia 202 directly onto the touch sensitive screen205, either at the beginning, middle, or end of an already establishedprocess.

Another embodiment shown in FIG. 5e is specific to a layered touchsensor, such as a PCT screen, as an underlay solution. In such anembodiment, the photoluminescent indicia 202 are disposed on the samelayer 206 as the component or components including the matrix ofelectrodes, and then the screen may be integrated to as normal intoother devices.

Yet another option is shown in FIG. 5f , which shows the digitizer panelas an underlay between the cover lens of a touch sensitive screen 205(in this case a visible light transmissive PCT screen) and the layer 206that includes the matrix of electrodes, by forming photoluminescentindicia 202 on a visible-light-transparent substrate 200, which couldcomprise materials such as PET or MOF, and adhering this to the coverglass using a variety of methods known to the art. The resultant stackcan then be adhered to the digitizer panel using methods known in theart (for example, by laminating with optically clear adhesive 201).

EXAMPLE

This example describes the set-up that may be used to demonstratephotoluminescent indicia that can be viewed at various stylus angles. Itconsists of an illumination source, a photoluminescent medium, and animage sensor placed behind a suitable filter. The illumination sourcecomprised a light emitting diode (UVXTZ-400-15 supplied by BIVAR, Inc,of Irving, Calif.), powered with 20 mA of current and placed at adistance of about 3 cm from the substrate on which a luminescent dye wasprinted to form position-unique photoluminescent indicia. The diode hada spectral emission centered at about 400 nm. Light from the diode wasincident on the printed fluorescent dye IRF820A and causedphotoluminescent emission. The photoluminescent material comprisedIRF820A, which was purchased from QCR Solutions Corp. of Port SaintLucie, Fla. It came in a powder form with a quantum efficiency quoted at0.2 for fluorescence between 700 nm and 1000 nm.

As an initial experiment, the fluorescent particles were dispersed atroom temperature into OP2001 Matte Varnish resin at a concentration of0.5% and manually deposited onto a small sample of MOF using atoothpick. The OP2001 Matte Varnish was purchased from the series for UVFlexo Varnishes of Nazdar Company of Shawnee, Kans. The MOF substratecomprised PR90EX manufactured by 3M Corporation of St. Paul, Minn. Thisparticular MOF was high in transmission over the visible range and washighly reflective beyond 850 nm. An optical filter was placed in thepath of the light entering a CCD image sensor that was used as animaging tool in the stylus. The optical filter was a long pass filter,comprising Clarex NIR-75N, supplied by Astra Products of Baldwin, N.Y.This filter suppressed transmission of light wavelengths below 750 nm.As a result, specular reflection from the diode that obscured someindicia in was largely suppressed from reaching the image sensor, whileIR light emitted by the fluorescent dye was detected by the imagesensor, resulting in a clear image of position-unique photoluminescentindicia on the substrate.

In a further experiment, an illumination source comprised a lightemitting diode (L750-04AU supplied by Marubeni America Corporation ofSanta Clara Calif.), powered with 20 mA of current and placed at adistance of about 3 cm from the substrate onto which a fluorescent dyewas printed to form position-unique photoluminescent indicia. The diodehad a spectral emission centered at about 750 nm. Light from the diodewas incident on the printed fluorescent dye EviDot Snake Eyes and causedphotoluminescent emission. The photoluminescent material comprisedEviDot Snake Eyes, which was purchased from Evident Technologies of TroyN.Y. It came in a liquid form of quantum dots in toluene with a quantumefficiency quoted at 0.3 for fluorescence between 400 nm and 1000 nm. Asan initial experiment, the fluorescent particles were dispersed at roomtemperature into Integrity 1100D and manually deposited onto a smallsample of PET using screen-printing techniques. The Integrity 1100D waspurchased from Hexion Specialty Chemicals of Columbus Ohio by EvidentTechnologies. The PET substrate comprised ST505 manufactured by DupontTeijin Films of Chester Va. This particular PET is a clear, heatstabilized polyester film which is pre-treated on both sides forimproved adhesion. An optical filter was placed in the path of the lightentering a CCD image sensor that was used as an imaging tool in thestylus. The optical filter was a long pass filter, comprising ClarexNIR-85N, supplied by Astra Products of Baldwin, N.Y. This filtersuppressed transmission of light wavelengths below 850 nm. As a result,specular reflection from the diode that obscured some indicia in waslargely suppressed from reaching the image sensor, while IR lightemitted by the fluorescent dye was detected by the image sensor,resulting in a clear image of position-unique photoluminescent indiciaon the substrate. These results were achieved at various stylus angles.

The term stylus used herein may include a device that may be movedrelative to a digitizer surface. The stylus may have a shape similar toa pen, or a computer mouse, or any shape. Multiple styli may be usedsimultaneously on a digitizer surface as writing devices, cursor controldevices, or game pieces, for example. The stylus may be moved manuallyor by a mechanical device or a machine. The digitizer surface may beplanar, cylindrical, spherical, or any shape.

Unless otherwise indicated, all numbers expressing quantities,measurement of properties, and so forth used in the specification andclaims are to be understood as being modified by the term “about”.Accordingly, unless indicated to the contrary, the numerical parametersset forth in the specification and claims are approximations that canvary depending on the desired properties sought to be obtained by thoseskilled in the art utilizing the teachings of the present application.Not as an attempt to limit the application of the doctrine ofequivalents to the scope of the claims, each numerical parameter shouldat least be construed in light of the number of reported significantdigits and by applying ordinary rounding techniques. Notwithstandingthat the numerical ranges and parameters setting forth the broad scopeof the invention are approximations, to the extent any numerical valuesare set forth in specific examples described herein, they are reportedas precisely as reasonably possible. Any numerical value, however, maywell contain errors associated with testing or measurement limitations.

Various modifications and alterations of this invention will be apparentto those skilled in the art without departing from the spirit and scopeof this invention, and it should be understood that this invention isnot limited to the illustrative embodiments set forth herein. Forexample, the reader should assume that features of one disclosedembodiment can also be applied to all other disclosed embodiments unlessotherwise indicated. It should also be understood that all U.S. patents,patent application publications, and other patent and non-patentdocuments referred to herein are incorporated by reference, to theextent they do not contradict the foregoing disclosure.

The invention claimed is:
 1. A method of calibrating a digitizer systemhaving a substrate with photoluminescent indicia that uniquely identifylocal areas of a substrate, with a display, the method of calibratingthe digitizer system with the display comprising: providing a processorcommunicatively coupled to the display and an electronic sensing device,wherein the processor is adapted to control images displayed on thedisplay, wherein the processor causes a first localized area of thedisplay to radiate, and wherein the electronic sensing device is astylus having a field-of-view and comprising different first and secondlight sources, the stylus being configured to operate in a plurality ofmodes, the plurality of modes comprising: a first mode for readingphotoluminescent indicia wherein the first light source is turned offand the second light source is turned on; a second mode for readinglight absorbing indicia on a light diffusing or reflective substratewherein the first light source is turned on and the second light sourceis turned off; and a third mode for reading indicia displayed on thedisplay wherein the first and second light sources are turned off; andreceiving, into the electronic sensing device, radiation from the firstlocalized area of the display and radiation emitted from aphotoluminescent indicium associated with the first localized area. 2.The method of claim 1, wherein the radiation from the first localizedarea of the display is received simultaneously with the radiationemitted from the photoluminescent indicium associated with the firstlocalized area, and wherein the radiation emitted from thephotoluminescent indicium is luminesced radiation.
 3. The method ofclaim 1, wherein the radiation from the first localized area of thedisplay and the radiation emitted from the photoluminescent indicium areboth within the stylus' field-of-view.
 4. The method of claim 1, furthercomprising: using the processor, associating the first localized area ofthe display with a first localized area of photoluminescent indicia. 5.The method of claim 4, further comprising: outputting calibration data,which represents the associated first localized area of the display andthe photoluminescent indicia associated with the first localized area ofphotoluminescent indicia.
 6. The method of claim 5, further comprising:receiving, into the electronic sensing device, radiation from a secondlocalized area of the display and radiation emitted from aphotoluminescent indicium associated with the second localized area. 7.The method of claim 5, further comprising: using the processor,associating the second localized area of the display with a secondlocalized area of photoluminescent indicia.
 8. The method of claim 7,wherein the calibration data further represents the associated secondlocalized area of the display with the photoluminescent indiciaassociated with the second localized area of photoluminescent indicia.9. The method of claim 1, further comprising: illuminating thephotoluminescent indicium associated with the first localized area withexcitation radiation having a wavelength range that causes the indiciumto luminesce in a wavelength range to which the electronic sensingdevice is sensitive.
 10. The method of claim 9 further comprisingplacing the electronic sensing device into the third mode, wherein theillumination is provided by the radiation from the first localized areaof the display.
 11. The method of claim 9 further comprising placing theelectronic sensing device into the first mode, wherein the illuminationis provided from the second light source.
 12. The method of claim 1,wherein the display comprises display pixels, and causing a firstlocalized area of the display to radiate comprises activating one ormore pixels of the display.
 13. The method of claim 12, wherein the oneor more pixels of the display comprises more than one pixel, and themore than one pixels are adjacent to one another.
 14. The method ofclaim 1, wherein the radiation from a first localized area of thedisplay is time modulated.
 15. A digitizer system comprising: anelectronically addressable display; a substrate having photoluminescentindicia that uniquely define local areas of the substrate, wherein thesubstrate is coupled to the display; a sensing unit that senses thephotoluminescent indicia; a processor communicatively coupled to thedisplay and the sensing unit, the processor adapted to control imagesdisplayed on the display; wherein the processor causes a first localizedarea of the display to radiate while receiving signals indicative ofradiation input received from the sensing unit sensing at least onephotoluminescent indicium associated with the first localized area;wherein the processor computes calibration data based on the receivedsignals, the calibration data associating localized areas of thesubstrate to the display; and wherein the sensing unit is a stylushaving a field-of-view and comprising different first and second lightsources, the stylus being configured to operate in a plurality of modes,the plurality of modes comprising: a first mode for readingphotoluminescent indicia wherein the first light source is turned offand the second light source is turned on; a second mode for readinglight absorbing indicia on a light diffusing or reflective substratewherein the first light source is turned on and the second light sourceis turned off and a third mode for reading indicia displayed on thedisplay wherein the first and second light sources are turned off. 16.The system of claim 15, wherein the radiation from the first localizedarea of the display is received by the sensing unit simultaneously withthe radiation from the at least one photoluminescent indicium associatedwith the first localized area.
 17. The system of claim 15, wherein theradiation from the first localized area of the display and the radiationfrom the photoluminescent indicium are both within the stylus'field-of-view.
 18. The system of claim 15, wherein the processor furtherassociates the first localized area of the display with the firstlocalized area of photoluminescent indicia.
 19. The system of claim 15,wherein the processor is adapted to output the calibration data.
 20. Thesystem of claim 15, wherein the processor causes a second localized areaof the display to radiate while receiving signals indicative ofradiation input received from the sensing unit sensing at least onephotoluminescent indicium associated with the second localized area. 21.The system of claim 15, wherein the system is adapted to illuminate theat least one photoluminescent indicium associated with the firstlocalized area with a radiation having a wavelength range that causesthe indicium to luminesce to emit radiation in a wavelength range towhich the sensing unit is sensitive.
 22. The system of claim 21, whereinthe illumination is provided by the radiation from the first localizedarea of the display.
 23. The system of claim 21, wherein theillumination is provided from the second light source.
 24. The system ofclaim 15, wherein the display comprises display pixels, and causing afirst localized area of the display to radiate comprises activating oneor more pixels of the display.
 25. The system of claim 24, wherein theone or more pixels of the display comprises more than one pixel, and themore than one pixels are adjacent to one another.
 26. The system ofclaim 15, wherein the radiation from a first localized area of thedisplay is time modulated.